Imaging dimensions of the plenoptic function has been a long-standing goal [Adelson and Bergen 1991]. Access to the full properties of the incident light on a sensor, e.g. the direction, spectrum, the temporal variation, the polarization and other properties has a large number of applications in scientific imaging, industrial quality control, remote sensing, computer vision, and computer graphics.
Numerous specialized devices, ranging from space-borne imagers to microscope cameras, exist for classic multispectral and polarization imaging. More recently high dynamic range imaging and light-field capture have become a major focus in computer graphics. In order to gain access to these physical dimensions of an image, the light integration has to be adapted.
In a temporal multiplexing approach, an image stack is recorded and filters of different exposures are placed in the light path. This approach can only be applied to static or quasi-static scenes. The latter requires a registration of the individual images which is a difficult problem in itself. In a hardware parallel acquisition approach, the optical image is multiplied by means of a beam-splitter arrangement and projected onto different sensor units that are spatially de-localized. Different optical pre-filters can be inserted into the different optical light paths. This arrangement allows for dynamic scenes to be imaged. It comes, however, at the price of large, expensive, and bulky setups that have to be custom built. Further, synchronization and radiometric calibration of the different sensors with respect to each other is another problematic aspect. Finally, in a spatial multiplexing approach, a single sensor unit is employed where every pixel is associated with a different optical pre-filter. This design allows a single-exposure (snapshot) retrieval. Its most familiar application is color imaging via a color filter array. Presently, this requires custom sensor designs or the permanent modification of the standard sensors.